Applications of Bayesian Knowledge Tracing to the Curation of Educational Videos

With the popularity of MOOCs and other online learning platforms such as Khan Academy, the role of online education has continued to increase in relation to that of traditional on-campus instruction. At the same time, the need for analytical methods suited for the uniquely large and diverse populations that they serve has grown apace. In particular, as instructors and creators of online educational content grapple with these complex issues, the imperfect transfer of traditional informal, frequently affect-oriented methods of content iteration becomes clear. The need for additional quantitative tools for evaluating course content, taken alongside the opportunity presented by the scope and size of the data associated with such large enrollment courses, poses an interesting problem for analysis. Rather than tackle the problem of evaluating large educational units such as entire online courses, our work approaches a smaller problem: exploring a framework for evaluating more granular educational units, in this case, short educational videos. We have chosen to leverage an adaptation of traditional Bayesian Knowledge Tracing (BKT), intended to evaluate the usage of video content in addition to assessment activity. By exploring the change in performance when alternately including or omitting video activity, we suggest a metric for determining the relevance of videos to associated assessments. This sort of evaluation is important for many reasons: struggling students can be pointed toward maximally efficacious resources, instructors can identify materials which may need adjustment, and courses as a whole can be better tuned to producing successful student outcomes. In order to provide an intuitive grounding for the validity of our results, we examine in detail the properties of videos that perform particularly well and those that do poorly, offering several case studies of the various data-sets included in this analysis. By proposing and demonstrating a new analytical approach to evaluating course content, we aim to move the promises offered by educational big data one step closer to practicable reality.